{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X4cRE8IbIrIV"
      },
      "source": [
        "本文涉及的jupter notebook在[篇章4代码库中](https://github.com/datawhalechina/learn-nlp-with-transformers/tree/main/docs/%E7%AF%87%E7%AB%A04-%E4%BD%BF%E7%94%A8Transformers%E8%A7%A3%E5%86%B3NLP%E4%BB%BB%E5%8A%A1)。\n",
        "\n",
        "如果您在colab上打开这个jupyter笔记本，您需要安装🤗Trasnformers和🤗datasets。具体命令如下（取消注释并运行，如果速度慢请切换国内源，加上第二行的参数）。\n",
        "\n",
        "在运行单元格之前，建议您按照本项目readme中提示，建立一个专门的python环境用于学习。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MOsHUjgdIrIW"
      },
      "outputs": [],
      "source": [
        "#! pip install datasets transformers \r\n",
        "# -i https://pypi.tuna.tsinghua.edu.cn/simple"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HFASsisvIrIb"
      },
      "source": [
        "如果您是在本地机器上打开这个jupyter笔记本，请确保您的环境安装了上述库的最新版本。\r\n",
        "\r\n",
        "您可以在[这里](https://github.com/huggingface/transformers/blob/master/examples/pytorch/multiple-choice/)找到这个jupyter笔记本的具体的python脚本文件，还可以通过分布式的方式使用多个gpu或tpu来微调您的模型。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rEJBSTyZIrIb"
      },
      "source": [
        "# 通过微调模型构建多选任务"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kTCFado4IrIc"
      },
      "source": [
        "在当前jupyter笔记本中，我们将说明如何通过微调任意[🤗Transformers](https://github.com/huggingface/transformers) 模型来构建多选任务，该任务是在给定的多个答案中选择最合理的一个。我们使用的数据集是[SWAG](https://www.aclweb.org/anthology/D18-1009/)，当然你也可以将预处理过程用于其他多选数据集或者你自己的数据。SWAG是一个关于常识推理的数据集，每个样本描述一种情况，然后给出四个可能的选项。\n",
        "\n",
        "这个jupyter笔记本可以运行在[model Hub](https://huggingface.co/models)中的任何模型上，只要该模型具有一个多选择头的版本。根据你的模型和你使用的GPU，你可能需要调整批大小，以避免显存不足的错误。设置好这两个参数之后，jupyter笔记本的其余部分就可以顺利运行了:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "zVvslsfMIrIh"
      },
      "outputs": [],
      "source": [
        "model_checkpoint = \"bert-base-uncased\"\r\n",
        "batch_size = 16"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "whPRbBNbIrIl"
      },
      "source": [
        "## 加载数据集"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W7QYTpxXIrIl"
      },
      "source": [
        "我们将使用[🤗Datasets](https://github.com/huggingface/datasets)库来下载数据。这一过程可以很容易地用函数`load_dataset`来完成。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "IreSlFmlIrIm"
      },
      "outputs": [],
      "source": [
        "from datasets import load_dataset, load_metric"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CKx2zKs5IrIq"
      },
      "source": [
        "`load_dataset` 将缓存数据集以避免下次运行时再次下载它。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 270,
          "referenced_widgets": [
            "69caab03d6264fef9fc5649bffff5e20",
            "3f74532faa86412293d90d3952f38c4a",
            "50615aa59c7247c4804ca5cbc7945bd7",
            "fe962391292a413ca55dc932c4279fa7",
            "299f4b4c07654e53a25f8192bd1d7bbd",
            "ad04ed1038154081bbb0c1444784dcc2",
            "7c667ad22b5740d5a6319f1b1e3a8097",
            "46c2b043c0f84806978784a45a4e203b",
            "80e2943be35f46eeb24c8ab13faa6578",
            "de5956b5008d4fdba807bae57509c393",
            "931db1f7a42f4b46b7ff8c2e1262b994",
            "6c1db72efff5476e842c1386fadbbdba",
            "ccd2f37647c547abb4c719b75a26f2de",
            "d30a66df5c0145e79693e09789d96b81",
            "5fa26fc336274073abbd1d550542ee33",
            "2b34de08115d49d285def9269a53f484",
            "d426be871b424affb455aeb7db5e822e",
            "160bf88485f44f5cb6eaeecba5e0901f",
            "745c0d47d672477b9bb0dae77b926364",
            "d22ab78269cd4ccfbcf70c707057c31b",
            "d298eb19eeff453cba51c2804629d3f4",
            "a7204ade36314c86907c562e0a2158b8",
            "e35d42b2d352498ca3fc8530393786b2",
            "75103f83538d44abada79b51a1cec09e",
            "f6253931d90543e9b5fd0bb2d615f73a",
            "051aa783ff9e47e28d1f9584043815f5",
            "0984b2a14115454bbb009df71c1cf36f",
            "8ab9dfce29854049912178941ef1b289",
            "c9de740e007141958545e269372780a4",
            "cbea68b25d6d4ba09b2ce0f27b1726d5",
            "5781fc45cf8d486cb06ed68853b2c644",
            "d2a92143a08a4951b55bab9bc0a6d0d3",
            "a14c3e40e5254d61ba146f6ec88eae25",
            "c4ffe6f624ce4e978a0d9b864544941a",
            "1aca01c1d8c940dfadd3e7144bb35718",
            "9fbbaae50e6743f2aa19342152398186",
            "fea27ca6c9504fc896181bc1ff5730e5",
            "940d00556cb849b3a689d56e274041c2",
            "5cdf9ed939fb42d4bf77301c80b8afca",
            "94b39ccfef0b4b08bf2fb61bb0a657c1",
            "9a55087c85b74ea08b3e952ac1d73cbe",
            "2361ab124daf47cc885ff61f2899b2af",
            "1a65887eb37747ddb75dc4a40f7285f2",
            "3c946e2260704e6c98593136bd32d921",
            "50d325cdb9844f62a9ecc98e768cb5af",
            "aa781f0cfe454e9da5b53b93e9baabd8",
            "6bb68d3887ef43809eb23feb467f9723",
            "7e29a8b952cf4f4ea42833c8bf55342f",
            "dd5997d01d8947e4b1c211433969b89b",
            "2ace4dc78e2f4f1492a181bcd63304e7",
            "bbee008c2791443d8610371d1f16b62b",
            "31b1c8a2e3334b72b45b083688c1a20c",
            "7fb7c36adc624f7dbbcb4a831c1e4f63",
            "0b7c8f1939074794b3d9221244b1344d",
            "a71908883b064e1fbdddb547a8c41743",
            "2f5223f26c8541fc87e91d2205c39995"
          ]
        },
        "id": "s_AY1ATSIrIq",
        "outputId": "fd0578d1-8895-443d-b56f-5908de9f1b6b"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Reusing dataset swag (/home/sgugger/.cache/huggingface/datasets/swag/regular/0.0.0/f9784740e0964a3c799d68cec0d992cc267d3fe94f3e048175eca69d739b980d)\n"
          ]
        }
      ],
      "source": [
        "datasets = load_dataset(\"swag\", \"regular\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "除此之外，你也可以从我们提供的[链接](https://gas.graviti.cn/dataset/datawhale/SWAG\n",
        ")下载数据并解压，将解压后的3个csv文件复制到到`docs/篇章4-使用Transformers解决NLP任务/datasets/swag`目录下，然后用下面的代码进行加载。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Using custom data configuration regular-2ab2d66f12115abf\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading and preparing dataset swag/regular (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) to ./datasets/swag/cache/swag/regular-2ab2d66f12115abf/0.0.0/a16ae67faa24f4cdd6d1fc6bfc09bdb6dc15771716221ff8bacbc6cc75533614...\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "                                         "
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Dataset swag downloaded and prepared to ./datasets/swag/cache/swag/regular-2ab2d66f12115abf/0.0.0/a16ae67faa24f4cdd6d1fc6bfc09bdb6dc15771716221ff8bacbc6cc75533614. Subsequent calls will reuse this data.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": []
        }
      ],
      "source": [
        "import os\n",
        "\n",
        "data_path = './datasets/swag/'\n",
        "cache_dir = os.path.join(data_path, 'cache')\n",
        "data_files = {'train': os.path.join(data_path, 'train.csv'), 'val': os.path.join(data_path, 'val.csv'), 'test': os.path.join(data_path, 'test.csv')}\n",
        "datasets = load_dataset(data_path, 'regular', data_files=data_files, cache_dir=cache_dir)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RzfPtOMoIrIu"
      },
      "source": [
        "`dataset`对象本身是[`DatasetDict`](https://huggingface.co/docs/datasets/package_reference/main_classes.html#datasetdict)，它包含用于训练、验证和测试集的键值对(`mnli`是一个特殊的例子，其中包含用于不匹配的验证和测试集的键值对)。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GWiVUF0jIrIv",
        "outputId": "35e3ea43-f397-4a54-c90c-f2cf8d36873e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "DatasetDict({\n",
              "    train: Dataset({\n",
              "        features: ['video-id', 'fold-ind', 'startphrase', 'sent1', 'sent2', 'gold-source', 'ending0', 'ending1', 'ending2', 'ending3', 'label'],\n",
              "        num_rows: 73546\n",
              "    })\n",
              "    validation: Dataset({\n",
              "        features: ['video-id', 'fold-ind', 'startphrase', 'sent1', 'sent2', 'gold-source', 'ending0', 'ending1', 'ending2', 'ending3', 'label'],\n",
              "        num_rows: 20006\n",
              "    })\n",
              "    test: Dataset({\n",
              "        features: ['video-id', 'fold-ind', 'startphrase', 'sent1', 'sent2', 'gold-source', 'ending0', 'ending1', 'ending2', 'ending3', 'label'],\n",
              "        num_rows: 20005\n",
              "    })\n",
              "})"
            ]
          },
          "execution_count": 5,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "datasets"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u3EtYfeHIrIz"
      },
      "source": [
        "To access an actual element, you need to select a split first, then give an index:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "X6HrpprwIrIz",
        "outputId": "d7670bc0-42e4-4c09-8a6a-5c018ded7d95"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'ending0': 'passes by walking down the street playing their instruments.',\n",
              " 'ending1': 'has heard approaching them.',\n",
              " 'ending2': \"arrives and they're outside dancing and asleep.\",\n",
              " 'ending3': 'turns the lead singer watches the performance.',\n",
              " 'fold-ind': '3416',\n",
              " 'gold-source': 'gold',\n",
              " 'label': 0,\n",
              " 'sent1': 'Members of the procession walk down the street holding small horn brass instruments.',\n",
              " 'sent2': 'A drum line',\n",
              " 'startphrase': 'Members of the procession walk down the street holding small horn brass instruments. A drum line',\n",
              " 'video-id': 'anetv_jkn6uvmqwh4'}"
            ]
          },
          "execution_count": 6,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "datasets[\"train\"][0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WHUmphG3IrI3"
      },
      "source": [
        "为了了解数据是什么样子的，下面的函数将显示数据集中随机选取的一些示例。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "i3j8APAoIrI3"
      },
      "outputs": [],
      "source": [
        "from datasets import ClassLabel\r\n",
        "import random\r\n",
        "import pandas as pd\r\n",
        "from IPython.display import display, HTML\r\n",
        "\r\n",
        "def show_random_elements(dataset, num_examples=10):\r\n",
        "    assert num_examples <= len(dataset), \"Can't pick more elements than there are in the dataset.\"\r\n",
        "    picks = []\r\n",
        "    for _ in range(num_examples):\r\n",
        "        pick = random.randint(0, len(dataset)-1)\r\n",
        "        while pick in picks:\r\n",
        "            pick = random.randint(0, len(dataset)-1)\r\n",
        "        picks.append(pick)\r\n",
        "    \r\n",
        "    df = pd.DataFrame(dataset[picks])\r\n",
        "    for column, typ in dataset.features.items():\r\n",
        "        if isinstance(typ, ClassLabel):\r\n",
        "            df[column] = df[column].transform(lambda i: typ.names[i])\r\n",
        "    display(HTML(df.to_html()))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SZy5tRB_IrI7",
        "outputId": "ba8f2124-e485-488f-8c0c-254f34f24f13"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>ending0</th>\n",
              "      <th>ending1</th>\n",
              "      <th>ending2</th>\n",
              "      <th>ending3</th>\n",
              "      <th>fold-ind</th>\n",
              "      <th>gold-source</th>\n",
              "      <th>label</th>\n",
              "      <th>sent1</th>\n",
              "      <th>sent2</th>\n",
              "      <th>startphrase</th>\n",
              "      <th>video-id</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>are seated on a field.</td>\n",
              "      <td>are skiing down the slope.</td>\n",
              "      <td>are in a lift.</td>\n",
              "      <td>are pouring out in a man.</td>\n",
              "      <td>16668</td>\n",
              "      <td>gold</td>\n",
              "      <td>1</td>\n",
              "      <td>A man is wiping the skiboard.</td>\n",
              "      <td>Group of people</td>\n",
              "      <td>A man is wiping the skiboard. Group of people</td>\n",
              "      <td>anetv_JmL6BiuXr_g</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>performs stunts inside a gym.</td>\n",
              "      <td>shows several shopping in the water.</td>\n",
              "      <td>continues his skateboard while talking.</td>\n",
              "      <td>is putting a black bike close.</td>\n",
              "      <td>11424</td>\n",
              "      <td>gold</td>\n",
              "      <td>0</td>\n",
              "      <td>The credits of the video are shown.</td>\n",
              "      <td>A lady</td>\n",
              "      <td>The credits of the video are shown. A lady</td>\n",
              "      <td>anetv_dWyE0o2NetQ</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>is emerging into the hospital.</td>\n",
              "      <td>are strewn under water at some wreckage.</td>\n",
              "      <td>tosses the wand together and saunters into the marketplace.</td>\n",
              "      <td>swats him upside down.</td>\n",
              "      <td>15023</td>\n",
              "      <td>gen</td>\n",
              "      <td>1</td>\n",
              "      <td>Through his binoculars, someone watches a handful of surfers being rolled up into the wave.</td>\n",
              "      <td>Someone</td>\n",
              "      <td>Through his binoculars, someone watches a handful of surfers being rolled up into the wave. Someone</td>\n",
              "      <td>lsmdc3016_CHASING_MAVERICKS-6791</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>spies someone sitting below.</td>\n",
              "      <td>opens the fridge and checks out the photo.</td>\n",
              "      <td>puts a little sheepishly.</td>\n",
              "      <td>staggers up to him.</td>\n",
              "      <td>5475</td>\n",
              "      <td>gold</td>\n",
              "      <td>3</td>\n",
              "      <td>He tips it upside down, and its little umbrella falls to the floor.</td>\n",
              "      <td>Back inside, someone</td>\n",
              "      <td>He tips it upside down, and its little umbrella falls to the floor. Back inside, someone</td>\n",
              "      <td>lsmdc1008_Spider-Man2-75503</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>carries her to the grave.</td>\n",
              "      <td>laughs as someone styles her hair.</td>\n",
              "      <td>sets down his glass.</td>\n",
              "      <td>stares after her then trudges back up into the street.</td>\n",
              "      <td>6904</td>\n",
              "      <td>gen</td>\n",
              "      <td>1</td>\n",
              "      <td>Someone kisses her smiling daughter on the cheek and beams back at the camera.</td>\n",
              "      <td>Someone</td>\n",
              "      <td>Someone kisses her smiling daughter on the cheek and beams back at the camera. Someone</td>\n",
              "      <td>lsmdc1028_No_Reservations-83242</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>stops someone and sweeps all the way back from the lower deck to join them.</td>\n",
              "      <td>is being dragged towards the monstrous animation.</td>\n",
              "      <td>beats out many events at the touch of the sword, crawling it.</td>\n",
              "      <td>reaches into a pocket and yanks open the door.</td>\n",
              "      <td>14089</td>\n",
              "      <td>gen</td>\n",
              "      <td>1</td>\n",
              "      <td>But before he can use his wand, he accidentally rams it up the troll's nostril.</td>\n",
              "      <td>The angry troll</td>\n",
              "      <td>But before he can use his wand, he accidentally rams it up the troll's nostril. The angry troll</td>\n",
              "      <td>lsmdc1053_Harry_Potter_and_the_philosophers_stone-95867</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>sees someone's name in the photo.</td>\n",
              "      <td>gives a surprised look.</td>\n",
              "      <td>kneels down and touches his ripped specs.</td>\n",
              "      <td>spies on someone's clock.</td>\n",
              "      <td>8407</td>\n",
              "      <td>gen</td>\n",
              "      <td>1</td>\n",
              "      <td>Someone keeps his tired eyes on the road.</td>\n",
              "      <td>Glancing over, he</td>\n",
              "      <td>Someone keeps his tired eyes on the road. Glancing over, he</td>\n",
              "      <td>lsmdc1024_Identity_Thief-82693</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>stops as someone speaks into the camera.</td>\n",
              "      <td>notices how blue his eyes are.</td>\n",
              "      <td>is flung out of the door and knocks the boy over.</td>\n",
              "      <td>flies through the air, its a fireball.</td>\n",
              "      <td>4523</td>\n",
              "      <td>gold</td>\n",
              "      <td>1</td>\n",
              "      <td>Both people are knocked back a few steps from the force of the collision.</td>\n",
              "      <td>She</td>\n",
              "      <td>Both people are knocked back a few steps from the force of the collision. She</td>\n",
              "      <td>lsmdc0043_Thelma_and_Luise-68271</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>sits close to the river.</td>\n",
              "      <td>have pet's supplies and pets.</td>\n",
              "      <td>pops parked outside the dirt facility, sending up a car highway to catch control.</td>\n",
              "      <td>displays all kinds of power tools and website.</td>\n",
              "      <td>8112</td>\n",
              "      <td>gold</td>\n",
              "      <td>1</td>\n",
              "      <td>A guy waits in the waiting room with his pet.</td>\n",
              "      <td>A pet store and its van</td>\n",
              "      <td>A guy waits in the waiting room with his pet. A pet store and its van</td>\n",
              "      <td>anetv_9VWoQpg9wqE</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>the slender someone, someone turns on the light.</td>\n",
              "      <td>, someone gives them to her boss then dumps some alcohol into dough.</td>\n",
              "      <td>liquids from a bowl, she slams them drunk.</td>\n",
              "      <td>wags his tail as someone returns to the hotel room.</td>\n",
              "      <td>10867</td>\n",
              "      <td>gold</td>\n",
              "      <td>3</td>\n",
              "      <td>Inside a convenience store, she opens a freezer case.</td>\n",
              "      <td>Dolce</td>\n",
              "      <td>Inside a convenience store, she opens a freezer case. Dolce</td>\n",
              "      <td>lsmdc3090_YOUNG_ADULT-43871</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "show_random_elements(datasets[\"train\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "INu1Ebe_-MjA"
      },
      "source": [
        "数据集中的每个示例都有一个上下文，它是由第一个句子(字段`sent1`)和第二个句子的简介(字段`sent2`)组成。然后给出四种可能的结尾(字段`ending0`， `ending1`， `ending2`和`ending3`)，然后让模型从中选择正确的一个(由字段`label`表示)。下面的函数让我们更直观地看到一个示例:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hwBJ1Hxe-MjB"
      },
      "outputs": [],
      "source": [
        "def show_one(example):\r\n",
        "    print(f\"Context: {example['sent1']}\")\r\n",
        "    print(f\"  A - {example['sent2']} {example['ending0']}\")\r\n",
        "    print(f\"  B - {example['sent2']} {example['ending1']}\")\r\n",
        "    print(f\"  C - {example['sent2']} {example['ending2']}\")\r\n",
        "    print(f\"  D - {example['sent2']} {example['ending3']}\")\r\n",
        "    print(f\"\\nGround truth: option {['A', 'B', 'C', 'D'][example['label']]}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "setBrX1T-MjB",
        "outputId": "9d6e7a61-2591-47ed-9dab-9bb52574d84e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Context: Members of the procession walk down the street holding small horn brass instruments.\n",
            "  A - A drum line passes by walking down the street playing their instruments.\n",
            "  B - A drum line has heard approaching them.\n",
            "  C - A drum line arrives and they're outside dancing and asleep.\n",
            "  D - A drum line turns the lead singer watches the performance.\n",
            "\n",
            "Ground truth: option A\n"
          ]
        }
      ],
      "source": [
        "show_one(datasets[\"train\"][0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "K9pwSZwl-MjB",
        "outputId": "3f18338a-065d-43dd-b1e7-7551ee6eee09"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Context: Now it's someone's turn to rain blades on his opponent.\n",
            "  A - Someone pats his shoulder and spins wildly.\n",
            "  B - Someone lunges forward through the window.\n",
            "  C - Someone falls to the ground.\n",
            "  D - Someone rolls up his fast run from the water and tosses in the sky.\n",
            "\n",
            "Ground truth: option C\n"
          ]
        }
      ],
      "source": [
        "show_one(datasets[\"train\"][15])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n9qywopnIrJH"
      },
      "source": [
        "## 数据预处理"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YVx71GdAIrJH"
      },
      "source": [
        "在将这些文本输入到模型之前，我们需要对它们进行预处理。这是由🤗transformer的`Tokenizer`完成的，正如它的名字所暗示的那样，它将输入表示为一系列token，然后通过查找预训练好的词汇表，将它们转换为相应的id。最后转换成模型所期望的格式，同时生成模型所需的其他输入。\n",
        "\n",
        "为了做到这一切，我们使用`AutoTokenizer`的`from_pretrained`方法实例化我们的tokenizer，它将确保:\n",
        "\n",
        "-我们得到一个对应于我们想要使用的模型架构的tokenizer，\n",
        "-我们下载好了预训练这个特定模型时使用的词表。\n",
        "\n",
        "同时，该词表将被缓存，因此下次运行时不会再次下载它。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eXNLu_-nIrJI"
      },
      "outputs": [],
      "source": [
        "from transformers import AutoTokenizer\r\n",
        "    \r\n",
        "tokenizer = AutoTokenizer.from_pretrained(model_checkpoint, use_fast=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vl6IidfdIrJK"
      },
      "source": [
        "我们将`use_fast=True`作为参数入，以使用🤗tokenizers库中的一个快速tokenizer(它由Rust支持的)。这些快速tokenizer几乎适用于所有模型，但如果您在前面的调用中出现错误，请删除该参数。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rowT4iCLIrJK"
      },
      "source": [
        "你可以直接在一个句子或一个句子对上调用这个tokenizer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "a5hBlsrHIrJL",
        "outputId": "acdaa98a-a8cd-4a20-89b8-cc26437bbe90"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'input_ids': [101, 7592, 1010, 2023, 2028, 6251, 999, 102, 1998, 2023, 6251, 3632, 2007, 2009, 1012, 102], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}"
            ]
          },
          "execution_count": 13,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "tokenizer(\"Hello, this one sentence!\", \"And this sentence goes with it.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qo_0B1M2IrJM"
      },
      "source": [
        "根据您选择的模型，您将在上面单元格返回的字典中看到不同的键值对。它们对于我们在这里所做的并不重要，只需要知道它们是我们稍后实例化的模型所需要的。如果您对此感兴趣，可以在[本教程](https://huggingface.co/transformers/preprocessing.html)中了解更多关于它们的信息。\n",
        "\n",
        "如下面的字典所示，为了对数据集进行预处理，我们需要知道包含句子的列的名称:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2C0hcmp9IrJQ"
      },
      "source": [
        "我们可以写一个函数来预处理我们的样本。在调用tokenizer之前，最棘手的部分是将所有可能的句子对放在两个大列表中，然后将结果拉平，以便每个示例有四个输入id、注意力掩码等。\n",
        "\n",
        "当调用`tokenizer`时，我们传入参数`truncation=True`。这将确保比所选模型所能处理的更长的输入将被截断为模型所能接受的最大长度。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vc0BSBLIIrJQ"
      },
      "outputs": [],
      "source": [
        "ending_names = [\"ending0\", \"ending1\", \"ending2\", \"ending3\"]\r\n",
        "\r\n",
        "def preprocess_function(examples):\r\n",
        "    # Repeat each first sentence four times to go with the four possibilities of second sentences.\r\n",
        "    first_sentences = [[context] * 4 for context in examples[\"sent1\"]]\r\n",
        "    # Grab all second sentences possible for each context.\r\n",
        "    question_headers = examples[\"sent2\"]\r\n",
        "    second_sentences = [[f\"{header} {examples[end][i]}\" for end in ending_names] for i, header in enumerate(question_headers)]\r\n",
        "    \r\n",
        "    # Flatten everything\r\n",
        "    first_sentences = sum(first_sentences, [])\r\n",
        "    second_sentences = sum(second_sentences, [])\r\n",
        "    \r\n",
        "    # Tokenize\r\n",
        "    tokenized_examples = tokenizer(first_sentences, second_sentences, truncation=True)\r\n",
        "    # Un-flatten\r\n",
        "    return {k: [v[i:i+4] for i in range(0, len(v), 4)] for k, v in tokenized_examples.items()}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0lm8ozrJIrJR"
      },
      "source": [
        "This function works with one or several examples. In the case of several examples, the tokenizer will return a list of lists of lists for each key: a list of all examples (here 5), then a list of all choices (4) and a list of input IDs (length varying here since we did not apply any padding):\n",
        "\n",
        "这个函数可以使用一个或多个示例。在传入多个示例时，tokenizer将为每个键返回一个列表的列表：所有示例的列表(长度为5)，然后是所有选项的列表(长度为4)以及输入id的列表(长度不同，因为我们没有应用任何填充):"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EiY4Forv-MjE",
        "outputId": "d615d86b-f4b8-4a4a-910a-b66c32af7c4c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "5 4 [30, 25, 30, 28]\n"
          ]
        }
      ],
      "source": [
        "examples = datasets[\"train\"][:5]\n",
        "features = preprocess_function(examples)\n",
        "print(len(features[\"input_ids\"]), len(features[\"input_ids\"][0]), [len(x) for x in features[\"input_ids\"][0]])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DDaODKSp-MjE"
      },
      "source": [
        "让我们解码一下给定示例的输入:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "p001A0Dh-MjE",
        "outputId": "8fe7572c-dc30-4c72-bb24-91f107648e46"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['[CLS] a drum line passes by walking down the street playing their instruments. [SEP] members of the procession are playing ping pong and celebrating one left each in quick. [SEP]',\n",
              " '[CLS] a drum line passes by walking down the street playing their instruments. [SEP] members of the procession wait slowly towards the cadets. [SEP]',\n",
              " '[CLS] a drum line passes by walking down the street playing their instruments. [SEP] members of the procession makes a square call and ends by jumping down into snowy streets where fans begin to take their positions. [SEP]',\n",
              " '[CLS] a drum line passes by walking down the street playing their instruments. [SEP] members of the procession play and go back and forth hitting the drums while the audience claps for them. [SEP]']"
            ]
          },
          "execution_count": 16,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "idx = 3\n",
        "[tokenizer.decode(features[\"input_ids\"][idx][i]) for i in range(4)]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hNLHrkl--MjF"
      },
      "source": [
        "我们可以将它和之前生成的ground truth进行比较："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xaW_Re7j-MjF",
        "outputId": "13e7ed39-00b2-456f-e866-2d563db5ab15"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Context: A drum line passes by walking down the street playing their instruments.\n",
            "  A - Members of the procession are playing ping pong and celebrating one left each in quick.\n",
            "  B - Members of the procession wait slowly towards the cadets.\n",
            "  C - Members of the procession makes a square call and ends by jumping down into snowy streets where fans begin to take their positions.\n",
            "  D - Members of the procession play and go back and forth hitting the drums while the audience claps for them.\n",
            "\n",
            "Ground truth: option D\n"
          ]
        }
      ],
      "source": [
        "show_one(datasets[\"train\"][3])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zS-6iXTkIrJT"
      },
      "source": [
        "这似乎没问题。我们可以将这个函数应用到我们数据集的所有示例中，只需要使用我们之前创建的`dataset`对象的`map`方法。这将应用于`dataset`对象的所有切分的所有元素，所以我们的训练，验证和测试数据将以相同的方式进行预处理。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DDtsaJeVIrJT",
        "outputId": "aa4734bf-4ef5-4437-9948-2c16363da719"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Loading cached processed dataset at /home/sgugger/.cache/huggingface/datasets/swag/regular/0.0.0/f9784740e0964a3c799d68cec0d992cc267d3fe94f3e048175eca69d739b980d/cache-975c81cf12e5b7ac.arrow\n",
            "Loading cached processed dataset at /home/sgugger/.cache/huggingface/datasets/swag/regular/0.0.0/f9784740e0964a3c799d68cec0d992cc267d3fe94f3e048175eca69d739b980d/cache-d4806d63f1eaf5cd.arrow\n",
            "Loading cached processed dataset at /home/sgugger/.cache/huggingface/datasets/swag/regular/0.0.0/f9784740e0964a3c799d68cec0d992cc267d3fe94f3e048175eca69d739b980d/cache-258c9cd71b0182db.arrow\n"
          ]
        }
      ],
      "source": [
        "encoded_datasets = datasets.map(preprocess_function, batched=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "voWiw8C7IrJV"
      },
      "source": [
        "更好的是，结果会被🤗Datasets库自动缓存，以避免下次运行时在这一步上花费时间。🤗Datasets库通常足够智能，它可以检测传递给`map`的函数何时发生更改(此时不再使用缓存数据)。例如，它将检测您是否在第一个单元格中更改了任务并重新运行笔记本。当🤗Datasets使用缓存文件时，它提示相应的警告，你可以在调用`map`中传入`load_from_cache_file=False`从而不使用缓存文件，并强制进行预处理。\n",
        "\n",
        "请注意，我们传递了`batched=True`以批量对文本进行编码。这是为了充分利用我们前面加载的快速tokenizer的优势，它将使用多线程并发地处理批中的文本。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "545PP3o8IrJV"
      },
      "source": [
        "## 微调模型"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FBiW8UpKIrJW"
      },
      "source": [
        "现在我们的数据已经准备好了，我们可以下载预训练好的模型并对其进行微调。因为我们的任务是关于多项选择的，所以我们使用`AutoModelForMultipleChoice`类。与tokenizer一样，`from_pretrained`方法将为我们下载并缓存模型。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TlqNaB8jIrJW",
        "outputId": "84916cf3-6e6c-47f3-d081-032ec30a4132"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMultipleChoice: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']\n",
            "- This IS expected if you are initializing BertForMultipleChoice from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing BertForMultipleChoice from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "Some weights of BertForMultipleChoice were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.weight', 'classifier.bias']\n",
            "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
          ]
        }
      ],
      "source": [
        "from transformers import AutoModelForMultipleChoice, TrainingArguments, Trainer\n",
        "\n",
        "model = AutoModelForMultipleChoice.from_pretrained(model_checkpoint)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CczA5lJlIrJX"
      },
      "source": [
        "这个警告告诉我们，我们正在丢弃一些权重(`vocab_transform`和`vocab_layer_norm`层)，并随机初始化其他一些参数(`pre_classifier`和`classifier`层)。这是完全正常的情况，因为我们舍弃了在预训练模型时用于掩码语言建模的头，代之以一个新的多选头，并且我们没有其预训练好的权重，所以这个警告告诉我们使用这个模型来推理之前需要微调，而这正是我们要做的。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_N8urzhyIrJY"
      },
      "source": [
        "为了实例化一个`Trainer`，我们需要定义另外三个东西。最重要的是[`TrainingArguments`](https://huggingface.co/transformers/main_classes/trainer.html#transformers.TrainingArguments)，它是一个包含所有用于训练的属性的类。它需要传入一个文件夹名，用于保存模型的检查点，而所有其他参数都是可选的:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Bliy8zgjIrJY"
      },
      "outputs": [],
      "source": [
        "args = TrainingArguments(\n",
        "    \"test-glue\",\n",
        "    evaluation_strategy = \"epoch\",\n",
        "    learning_rate=5e-5,\n",
        "    per_device_train_batch_size=batch_size,\n",
        "    per_device_eval_batch_size=batch_size,\n",
        "    num_train_epochs=3,\n",
        "    weight_decay=0.01,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "km3pGVdTIrJc"
      },
      "source": [
        "在这里，我们设置在每个epoch的末尾进行评估，调整学习速率，使用在jupyter笔记本顶部定义的`batch_size`，并定制用于训练的epoch的数量，以及权重衰减。\n",
        "\n",
        "然后，我们需要告诉我们的`Trainer`如何从预处理的输入数据中构造批数据。我们还没有做任何填充，因为我们将填充每个批到批内的最大长度(而不是使用整个数据集的最大长度)。这将是*data collator*的工作。它接受示例的列表，并将它们转换为一个批(在我们的示例中，通过应用填充)。由于在库中没有data collator来处理我们的特定问题，这里我们根据`DataCollatorWithPadding`自行改编一个:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FuyAV8uh-MjH"
      },
      "outputs": [],
      "source": [
        "from dataclasses import dataclass\n",
        "from transformers.tokenization_utils_base import PreTrainedTokenizerBase, PaddingStrategy\n",
        "from typing import Optional, Union\n",
        "import torch\n",
        "\n",
        "@dataclass\n",
        "class DataCollatorForMultipleChoice:\n",
        "    \"\"\"\n",
        "    Data collator that will dynamically pad the inputs for multiple choice received.\n",
        "    \"\"\"\n",
        "\n",
        "    tokenizer: PreTrainedTokenizerBase\n",
        "    padding: Union[bool, str, PaddingStrategy] = True\n",
        "    max_length: Optional[int] = None\n",
        "    pad_to_multiple_of: Optional[int] = None\n",
        "\n",
        "    def __call__(self, features):\n",
        "        label_name = \"label\" if \"label\" in features[0].keys() else \"labels\"\n",
        "        labels = [feature.pop(label_name) for feature in features]\n",
        "        batch_size = len(features)\n",
        "        num_choices = len(features[0][\"input_ids\"])\n",
        "        flattened_features = [[{k: v[i] for k, v in feature.items()} for i in range(num_choices)] for feature in features]\n",
        "        flattened_features = sum(flattened_features, [])\n",
        "        \n",
        "        batch = self.tokenizer.pad(\n",
        "            flattened_features,\n",
        "            padding=self.padding,\n",
        "            max_length=self.max_length,\n",
        "            pad_to_multiple_of=self.pad_to_multiple_of,\n",
        "            return_tensors=\"pt\",\n",
        "        )\n",
        "        \n",
        "        # Un-flatten\n",
        "        batch = {k: v.view(batch_size, num_choices, -1) for k, v in batch.items()}\n",
        "        # Add back labels\n",
        "        batch[\"labels\"] = torch.tensor(labels, dtype=torch.int64)\n",
        "        return batch"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UUnusgMC-MjH"
      },
      "source": [
        "当传入一个示例的列表时，它会将大列表中的所有输入/注意力掩码等都压平，并传递给`tokenizer.pad`方法。这将返回一个带有大张量的字典(其大小为`(batch_size * 4) x seq_length`)，然后我们将其展开。\n",
        "\n",
        "我们可以在特征列表上检查data collator是否正常工作，在这里，我们只需要确保删除所有不被我们的模型接受的输入特征(这是`Trainer`自动为我们做的)："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "u2o1FcOy-MjH"
      },
      "outputs": [],
      "source": [
        "accepted_keys = [\"input_ids\", \"attention_mask\", \"label\"]\n",
        "features = [{k: v for k, v in encoded_datasets[\"train\"][i].items() if k in accepted_keys} for i in range(10)]\n",
        "batch = DataCollatorForMultipleChoice(tokenizer)(features)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VK-bctBG-MjH"
      },
      "source": [
        "再次强调，所有这些压平的、未压平的都可能是潜在错误的来源，所以让我们对输入进行另一个完整性检查："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vvkgo6Q6-MjI",
        "outputId": "7671e59e-4b18-47c1-f1be-6f0a6d08bee9"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['[CLS] someone walks over to the radio. [SEP] someone hands her another phone. [SEP] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD]',\n",
              " '[CLS] someone walks over to the radio. [SEP] someone takes the drink, then holds it. [SEP] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD]',\n",
              " '[CLS] someone walks over to the radio. [SEP] someone looks off then looks at someone. [SEP] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD]',\n",
              " '[CLS] someone walks over to the radio. [SEP] someone stares blearily down at the floor. [SEP] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD]']"
            ]
          },
          "execution_count": 25,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "[tokenizer.decode(batch[\"input_ids\"][8][i].tolist()) for i in range(4)]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EJTwH7ad-MjI",
        "outputId": "51698f2b-46dc-419f-e055-f7f31ba3068d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Context: Someone walks over to the radio.\n",
            "  A - Someone hands her another phone.\n",
            "  B - Someone takes the drink, then holds it.\n",
            "  C - Someone looks off then looks at someone.\n",
            "  D - Someone stares blearily down at the floor.\n",
            "\n",
            "Ground truth: option D\n"
          ]
        }
      ],
      "source": [
        "show_one(datasets[\"train\"][8])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7sZOdRlRIrJd"
      },
      "source": [
        "所有的都正常运行!\n",
        "\n",
        "最后要为`Trainer`定义如何根据预测计算评估指标。我们需要来定义一个函数，它将使用我们之前加载的`metric`，我们必须做的唯一预处理是取我们预测的logits的argmax："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UmvbnJ9JIrJd"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "\n",
        "def compute_metrics(eval_predictions):\n",
        "    predictions, label_ids = eval_predictions\n",
        "    preds = np.argmax(predictions, axis=1)\n",
        "    return {\"accuracy\": (preds == label_ids).astype(np.float32).mean().item()}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rXuFTAzDIrJe"
      },
      "source": [
        "然后，我们只需要将所有这些以及我们的数据集一起传入`Trainer`："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "imY1oC3SIrJf"
      },
      "outputs": [],
      "source": [
        "trainer = Trainer(\n",
        "    model,\n",
        "    args,\n",
        "    train_dataset=encoded_datasets[\"train\"],\n",
        "    eval_dataset=encoded_datasets[\"validation\"],\n",
        "    tokenizer=tokenizer,\n",
        "    data_collator=DataCollatorForMultipleChoice(tokenizer),\n",
        "    compute_metrics=compute_metrics,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CdzABDVcIrJg"
      },
      "source": [
        "现在，我们可以通过调用`train`方法来微调模型："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nxifCxew-MjJ",
        "outputId": "1c21b47d-c4f9-4a3f-9418-bab23c5b3589",
        "scrolled": true
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "        <style>\n",
              "            /* Turns off some styling */\n",
              "            progress {\n",
              "                /* gets rid of default border in Firefox and Opera. */\n",
              "                border: none;\n",
              "                /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "                background-size: auto;\n",
              "            }\n",
              "        </style>\n",
              "      \n",
              "      <progress value='6897' max='6897' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [6897/6897 23:49, Epoch 3/3]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: left;\">\n",
              "      <th>Epoch</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "      <th>Accuracy</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>0.154598</td>\n",
              "      <td>0.828017</td>\n",
              "      <td>0.766520</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>0.296633</td>\n",
              "      <td>0.667454</td>\n",
              "      <td>0.786814</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>0.111786</td>\n",
              "      <td>0.994927</td>\n",
              "      <td>0.789363</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "TrainOutput(global_step=6897, training_loss=0.19714653808275168)"
            ]
          },
          "execution_count": 35,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6IL_Yncb-MjJ"
      },
      "source": [
        "最后，不要忘记将你的模型[上传](https://huggingface.co/transformers/model_sharing.html)到[🤗 模型中心](https://huggingface.co/models)。"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "4.4-问答任务-多选问答",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.9"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "051aa783ff9e47e28d1f9584043815f5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0984b2a14115454bbb009df71c1cf36f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "info",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cbea68b25d6d4ba09b2ce0f27b1726d5",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c9de740e007141958545e269372780a4",
            "value": 1
          }
        },
        "0b7c8f1939074794b3d9221244b1344d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "160bf88485f44f5cb6eaeecba5e0901f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1a65887eb37747ddb75dc4a40f7285f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "info",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_aa781f0cfe454e9da5b53b93e9baabd8",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_50d325cdb9844f62a9ecc98e768cb5af",
            "value": 1
          }
        },
        "1aca01c1d8c940dfadd3e7144bb35718": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "info",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_940d00556cb849b3a689d56e274041c2",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_fea27ca6c9504fc896181bc1ff5730e5",
            "value": 1
          }
        },
        "2361ab124daf47cc885ff61f2899b2af": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "299f4b4c07654e53a25f8192bd1d7bbd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": "initial"
          }
        },
        "2ace4dc78e2f4f1492a181bcd63304e7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2b34de08115d49d285def9269a53f484": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2f5223f26c8541fc87e91d2205c39995": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "31b1c8a2e3334b72b45b083688c1a20c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2f5223f26c8541fc87e91d2205c39995",
            "placeholder": "​",
            "style": "IPY_MODEL_a71908883b064e1fbdddb547a8c41743",
            "value": " 4.39k/? [00:00&lt;00:00, 149kB/s]"
          }
        },
        "3c946e2260704e6c98593136bd32d921": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7e29a8b952cf4f4ea42833c8bf55342f",
            "placeholder": "​",
            "style": "IPY_MODEL_6bb68d3887ef43809eb23feb467f9723",
            "value": " 1063/0 [00:00&lt;00:00, 12337.52 examples/s]"
          }
        },
        "3f74532faa86412293d90d3952f38c4a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "46c2b043c0f84806978784a45a4e203b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "50615aa59c7247c4804ca5cbc7945bd7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "Downloading: ",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ad04ed1038154081bbb0c1444784dcc2",
            "max": 7826,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_299f4b4c07654e53a25f8192bd1d7bbd",
            "value": 7826
          }
        },
        "50d325cdb9844f62a9ecc98e768cb5af": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": "initial"
          }
        },
        "5781fc45cf8d486cb06ed68853b2c644": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5cdf9ed939fb42d4bf77301c80b8afca": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5fa26fc336274073abbd1d550542ee33": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "69caab03d6264fef9fc5649bffff5e20": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_50615aa59c7247c4804ca5cbc7945bd7",
              "IPY_MODEL_fe962391292a413ca55dc932c4279fa7"
            ],
            "layout": "IPY_MODEL_3f74532faa86412293d90d3952f38c4a"
          }
        },
        "6bb68d3887ef43809eb23feb467f9723": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6c1db72efff5476e842c1386fadbbdba": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2b34de08115d49d285def9269a53f484",
            "placeholder": "​",
            "style": "IPY_MODEL_5fa26fc336274073abbd1d550542ee33",
            "value": " 28.7k/? [00:00&lt;00:00, 571kB/s]"
          }
        },
        "745c0d47d672477b9bb0dae77b926364": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "Downloading: 100%",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a7204ade36314c86907c562e0a2158b8",
            "max": 376971,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d298eb19eeff453cba51c2804629d3f4",
            "value": 376971
          }
        },
        "75103f83538d44abada79b51a1cec09e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7c667ad22b5740d5a6319f1b1e3a8097": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7e29a8b952cf4f4ea42833c8bf55342f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7fb7c36adc624f7dbbcb4a831c1e4f63": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": "initial"
          }
        },
        "80e2943be35f46eeb24c8ab13faa6578": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_931db1f7a42f4b46b7ff8c2e1262b994",
              "IPY_MODEL_6c1db72efff5476e842c1386fadbbdba"
            ],
            "layout": "IPY_MODEL_de5956b5008d4fdba807bae57509c393"
          }
        },
        "8ab9dfce29854049912178941ef1b289": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d2a92143a08a4951b55bab9bc0a6d0d3",
            "placeholder": "​",
            "style": "IPY_MODEL_5781fc45cf8d486cb06ed68853b2c644",
            "value": " 8551/0 [00:00&lt;00:00, 25108.88 examples/s]"
          }
        },
        "931db1f7a42f4b46b7ff8c2e1262b994": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "Downloading: ",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d30a66df5c0145e79693e09789d96b81",
            "max": 4473,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ccd2f37647c547abb4c719b75a26f2de",
            "value": 4473
          }
        },
        "940d00556cb849b3a689d56e274041c2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "94b39ccfef0b4b08bf2fb61bb0a657c1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9a55087c85b74ea08b3e952ac1d73cbe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1a65887eb37747ddb75dc4a40f7285f2",
              "IPY_MODEL_3c946e2260704e6c98593136bd32d921"
            ],
            "layout": "IPY_MODEL_2361ab124daf47cc885ff61f2899b2af"
          }
        },
        "9fbbaae50e6743f2aa19342152398186": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_94b39ccfef0b4b08bf2fb61bb0a657c1",
            "placeholder": "​",
            "style": "IPY_MODEL_5cdf9ed939fb42d4bf77301c80b8afca",
            "value": " 1043/0 [00:00&lt;00:00, 13590.50 examples/s]"
          }
        },
        "a14c3e40e5254d61ba146f6ec88eae25": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1aca01c1d8c940dfadd3e7144bb35718",
              "IPY_MODEL_9fbbaae50e6743f2aa19342152398186"
            ],
            "layout": "IPY_MODEL_c4ffe6f624ce4e978a0d9b864544941a"
          }
        },
        "a71908883b064e1fbdddb547a8c41743": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a7204ade36314c86907c562e0a2158b8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aa781f0cfe454e9da5b53b93e9baabd8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ad04ed1038154081bbb0c1444784dcc2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bbee008c2791443d8610371d1f16b62b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "Downloading: ",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0b7c8f1939074794b3d9221244b1344d",
            "max": 1586,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_7fb7c36adc624f7dbbcb4a831c1e4f63",
            "value": 1586
          }
        },
        "c4ffe6f624ce4e978a0d9b864544941a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c9de740e007141958545e269372780a4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": "initial"
          }
        },
        "cbea68b25d6d4ba09b2ce0f27b1726d5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ccd2f37647c547abb4c719b75a26f2de": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": "initial"
          }
        },
        "d22ab78269cd4ccfbcf70c707057c31b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_75103f83538d44abada79b51a1cec09e",
            "placeholder": "​",
            "style": "IPY_MODEL_e35d42b2d352498ca3fc8530393786b2",
            "value": " 377k/377k [00:00&lt;00:00, 703kB/s]"
          }
        },
        "d298eb19eeff453cba51c2804629d3f4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": "initial"
          }
        },
        "d2a92143a08a4951b55bab9bc0a6d0d3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d30a66df5c0145e79693e09789d96b81": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d426be871b424affb455aeb7db5e822e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_745c0d47d672477b9bb0dae77b926364",
              "IPY_MODEL_d22ab78269cd4ccfbcf70c707057c31b"
            ],
            "layout": "IPY_MODEL_160bf88485f44f5cb6eaeecba5e0901f"
          }
        },
        "dd5997d01d8947e4b1c211433969b89b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_bbee008c2791443d8610371d1f16b62b",
              "IPY_MODEL_31b1c8a2e3334b72b45b083688c1a20c"
            ],
            "layout": "IPY_MODEL_2ace4dc78e2f4f1492a181bcd63304e7"
          }
        },
        "de5956b5008d4fdba807bae57509c393": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e35d42b2d352498ca3fc8530393786b2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f6253931d90543e9b5fd0bb2d615f73a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0984b2a14115454bbb009df71c1cf36f",
              "IPY_MODEL_8ab9dfce29854049912178941ef1b289"
            ],
            "layout": "IPY_MODEL_051aa783ff9e47e28d1f9584043815f5"
          }
        },
        "fe962391292a413ca55dc932c4279fa7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_46c2b043c0f84806978784a45a4e203b",
            "placeholder": "​",
            "style": "IPY_MODEL_7c667ad22b5740d5a6319f1b1e3a8097",
            "value": " 28.7k/? [00:00&lt;00:00, 652kB/s]"
          }
        },
        "fea27ca6c9504fc896181bc1ff5730e5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": "initial"
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}